Apparatus and methods for obstacle detection

ABSTRACT

Systems, apparatuses and methods for recognizing or detecting obstacles are provided. Passive infrared (PIR) sensors may be coupled to movable objects, such as unmanned aerial vehicles (UAVs). PIR sensors may detect and recognize obstacles such as humans and determine or calculate a distance to the obstacles. Based on the distance from the movable object to the obstacle, one or more flight response measures such as collision avoidance maneuvers may be effected or implemented.

CROSS-REFERENCE

This application is a continuation of International Application No.PCT/CN2015/078792, filed on May 12, 2015. The above-referencedapplication is hereby incorporated by reference.

BACKGROUND OF THE INVENTION

Movable objects, such as unmanned aerial vehicles (UAVs), can be usedfor performing surveillance, reconnaissance, and exploration tasks in awide variety of environments for both military and civilianapplications. Such movable objects may operate in close proximity toobstacles (e.g., objects or subjects such as human beings) and ininstances may come into contact with the obstacles. Collision withhumans, animals, or objects may lead to injury, damage (e.g., to objectsor the UAV) or failure of operations UAVs are tasked with. To avoidcollision, UAVs may include sensors configured for obstacle avoidance.

Existing approaches for obstacle avoidance may be less than optimal insome instances. For example, collision avoidance utilizing visionsensors may be disabled or limited in dim or dark light and may utilizecomplicated algorithms requiring costly CPUs. For example, collisionavoidance utilizing lidar sensors may deteriorate in an outdoorenvironment due to sunlight and may operate under a complex mechanismrequiring many parts (e.g., laser head, rotating device, camera,processors, etc).

SUMMARY

Embodiments disclosed herein provide apparatuses and methods ofrecognizing or detecting obstacles. In many embodiments, passiveinfrared (PIR) sensors may be provided on movable objects and may beused to detect or recognize obstacles. The obstacles may comprise anyobjects or subjects within an environment. PIR sensors may generate databased on thermal signals from the obstacle, and a distance from themovable object to the obstacle may be determined or calculated based onthe data. Based on factors such as the distance from the movable objectto the obstacle, an appropriate flight response measure may bedetermined. Advantageously, the approaches described herein may provideimproved and efficient collision avoidance for movable objects,especially with regards to obstacles with particular thermal signals(e.g., for subjects such as human beings).

Thus, in one aspect, a method of recognizing an obstacle is provided.The method comprises: receiving, with aid of a passive infrared sensoron board a movable object, one or more heat signals from the obstacle;calculating, with aid of a processor, a distance from the movable objectto the obstacle based on data from the passive infrared sensor; anddetermining, based on the distance, whether to effect a collisionavoidance maneuver for the movable object to avoid the obstacle.

In some embodiments, the method further comprises effecting thecollision avoidance maneuver when a determination is made to effect acollision avoidance maneuver based on the distance. In some embodiments,the method further comprises recognizing the obstacle based on the heatsignal detected using the passive infrared sensor. In some embodiments,the movable object is an unmanned aerial vehicle (UAV). In someembodiments, the passive infrared sensor receives the heat signal afterthe heat signal has passed through an optical element. In someembodiments, the optical element is a Fresnel lens. In some embodiments,the recognition of the obstacle includes differentiating the obstaclefrom different types of obstacles based on the heat signal. In someembodiments, the obstacle is a human, and the heat signal from the humanis recognizable as belonging to a human. In some embodiments, the heatsignal is about 10 um in wavelength. In some embodiments, thedetermination whether to effect the collision avoidance maneuver is alsomade based on a speed and direction of the movable object and/or theobstacle. In some embodiments, the collision avoidance maneuver includesbraking. In some embodiments, the collision avoidance maneuver includesaltering a direction of a moving course of the movable object. In someembodiments, the collision avoidance maneuver includes stopping one ormore propulsion units of the movable object. In some embodiments, thecollision avoidance maneuver includes deploying one or more airbags. Insome embodiments, the collision avoidance maneuver includes deployingone or more parachutes.

In another aspect, an apparatus for detecting an obstacle is provided.The apparatus comprises: a passive infrared sensor on board a movableobject, said passive infrared sensor configured to receive one or moreheat signals from the obstacle; and one or more processors, collectivelyor individually configured to: calculate a distance from the movableobject to the obstacle based on data from the passive infrared sensor;and determine whether to effect a collision avoidance maneuver for themovable object to avoid the obstacle based on the distance.

In some embodiments, the one or more processors are configured to effectthe collision avoidance maneuver when a determination is made to effecta collision avoidance maneuver based on the distance. In someembodiments, the one or more processors are configured to recognize theobstacle based on the one or more heat signals received by the passiveinfrared sensor. In some embodiments, the movable object is an unmannedaerial vehicle (UAV). In some embodiments, the passive infrared sensorreceives the heat signal after the heat signal has passed through anoptical element. In some embodiments, the optical element is a Fresnellens. In some embodiments, the one or more processors are configured todifferentiate the obstacle from different types of obstacles based onthe heat signal. In some embodiments, the obstacle is a human, and theheat signal from the human is recognizable as belonging to a human. Insome embodiments, the heat signal is about 10 um in wavelength. In someembodiments, the one or more processors are configured to determinewhether to effect the collision avoidance maneuver based on the distanceand on a speed and direction of the movable object and/or the obstacle.In some embodiments, the collision avoidance maneuver includes braking.In some embodiments, the collision avoidance maneuver includes alteringa direction of a moving course of the movable object. In someembodiments, the collision avoidance maneuver includes stopping one ormore propulsion units of the movable object. In some embodiments, thecollision avoidance maneuver includes deploying one or more airbags. Insome embodiments, the collision avoidance maneuver includes deployingone or more parachutes.

In another aspect, a method of recognizing an obstacle is provided. Themethod comprises: providing a plurality of passive infrared sensorson-board a movable object, each passive infrared sensor of saidplurality having a different field of view; receiving, with aid of atleast one passive infrared sensor of said plurality, a heat signal fromthe obstacle; recognizing the obstacle based on the heat signal receivedusing the passive infrared sensor; and determining, based on data fromthe passive infrared sensor, whether to effect a collision avoidancemaneuver for the movable object to avoid the obstacle.

In some embodiments, a collective field of view of the plurality ofinfrared sensors covers a 360 angle around the movable object. In someembodiments, the 360 angle covers a panoramic view of the lateral sidesof the movable object. In some embodiments, a collective field of viewof the plurality of passive infrared sensors covers an entire sphericalregion around the movable object. In some embodiments, the movableobject is an unmanned aerial vehicle (UAV). In some embodiments, themethod further comprises effecting the collision avoidance maneuver whena determination is made to effect a collision avoidance maneuver basedon a distance from the movable object to the obstacle. In someembodiments, the infrared sensor receives the heat signal after the heatsignal has passed through an optical element. In some embodiments, theoptical element is a Fresnel lens. In some embodiments, the recognitionof the obstacle includes differentiating the obstacle from differenttypes of obstacles based on the heat signal. In some embodiments, theobstacle is a human, and the heat signal from the human is recognizableas belonging to a human. In some embodiments, the heat signal is about10 um in wavelength. In some embodiments, the determination whether toeffect the collision avoidance maneuver is made based on a speed anddirection of the movable object and/or the obstacle. In someembodiments, the collision avoidance maneuver includes braking. In someembodiments, the collision avoidance maneuver includes altering adirection of a moving course of the movable object. In some embodiments,the collision avoidance maneuver includes stopping one or morepropulsion units of the movable object. In some embodiments, thecollision avoidance maneuver includes deploying one or more airbags. Insome embodiments, the collision avoidance maneuver includes deployingone or more parachutes.

In another aspect, an apparatus for detecting an obstacle is provided.The apparatus comprises: a plurality of passive infrared sensors onboard a movable object, each passive infrared sensor of said pluralityhaving a different field of view; and one or more processors,collectively or individually configured to: receive a signal from atleast one passive infrared sensor of said plurality indicative of a heatsignal from the obstacle; recognize the obstacle based on the heatsignal received using the passive infrared sensor; and determine whetherto effect a collision avoidance maneuver for the movable object to avoidthe obstacle.

In some embodiments, a collective field of view of the plurality ofinfrared sensors covers a 360 angle around the movable object. In someembodiments, the 360 angle covers a panoramic view of the lateral sidesof the movable object. In some embodiments, a collective field of viewof the plurality of passive infrared sensors covers an entire sphericalregion around the movable object. In some embodiments, the movableobject is an unmanned aerial vehicle (UAV). In some embodiments, the oneor more processors are configured to effect the collision avoidancemaneuver when a determination is made to effect a collision avoidancemaneuver based on a distance from the movable object to the obstacle. Insome embodiments, the infrared sensor receives the heat signal after theheat signal has passed through an optical element. In some embodiments,the optical element is a Fresnel lens. In some embodiments, the one ormore processors are configured to differentiate the obstacle fromdifferent types of obstacles based on the heat signal. In someembodiments, the obstacle is a human, and the heat signal from the humanis recognizable as belonging to a human. In some embodiments, the heatsignal is about 10 um in wavelength. In some embodiments, the one ormore processors are configured to determine whether to effect thecollision avoidance maneuver based on a speed and direction of themovable object and/or the obstacle. In some embodiments, the collisionavoidance maneuver includes braking. In some embodiments, the collisionavoidance maneuver includes altering a direction of a moving course ofthe movable object. In some embodiments, the collision avoidancemaneuver includes stopping one or more propulsion units of the movableobject. In some embodiments, the collision avoidance maneuver includesdeploying one or more airbags. In some embodiments, the collisionavoidance maneuver includes deploying one or more parachutes.

In another aspect, a method for responding to a target is provided. Themethod comprises: receiving, with aid of one or more passive infraredsensors on board a movable object, one or more heat signals from thetarget; recognizing the target based on the received heat signals;performing, with aid of one or more processors, one or more flightresponse measures based on the recognized target.

In some embodiments, the one or more flight response measures areperformed automatically with the aid of the one or more processors. Insome embodiments, the method further comprises determining one or moreappropriate flight response measures prior to performing one or moreflight response measures based on the recognized target. In someembodiments, the method comprises determining a distance from themovable object to the target. In some embodiments, the distance iscompared to a threshold distance. In some embodiments, the one or moreflight response measures depend at least partly on the distance. In someembodiments, the one or more flight response comprises tracking thetarget. In some embodiments, tracking the target comprises following thetarget. In some embodiments, the target is followed at a predetermineddistance. In some embodiments, the one or more flight response measurescomprise sending an alert to an operator of the movable object. In someembodiments, the one or more flight response measures comprisemaintaining a predetermined distance from the target. In someembodiments, the one or more flight response measures comprisetriggering an imaging device to capture images. In some embodiments, theone or more flight response measures comprise a collision avoidancemaneuver. In some embodiments, the collision avoidance maneuvercomprises braking, altering a direction of a moving course of themovable object, stopping one or more propulsion units of the movableobject, deploying one or more airbags, and/or deploying one or moreparachutes. In some embodiments, the one or more passive infraredsensors individually or collectively have a 360° horizontal field ofview around the movable object. In some embodiments, each of the one ormore passive infrared sensors has a detection range of 5 m. In someembodiments, recognizing the target based on the detected heat signalcomprises differentiating the target from different types of targetsbased on the heat signal. In some embodiments, the target is a human,and the heat signal from the human is recognizable as belonging to ahuman. In some embodiments, the heat signal is about 8-12 um inwavelength.

In another aspect, a system for responding to a target is provided. Thesystem comprises: one or more passive infrared sensors on board amovable object, said passive infrared sensor configured to receive oneor more heat signals from the target; and one or more processors,collectively or individually configured to: recognize the target basedon the received heat signals; and perform one or more flight responsemeasures based on the recognized target.

In some embodiments, the one or more processors are configured toautomatically perform one or more flight response measures based on therecognized target. In some embodiments, the one or more processors areconfigured to determine one or more appropriate flight responsemeasures. In some embodiments, the one or more processors are configuredto determine a distance from the movable object to the target. In someembodiments, the one or more processors are configured to compare thedistance to a threshold distance. In some embodiments, the one or moreflight response measures depend at least partly on the distance. In someembodiments, the one or more flight response measure comprises trackingthe target. In some embodiments, tracking the target comprises followingthe target. In some embodiments, the target is followed at apredetermined distance. In some embodiments, the one or more flightresponse measures comprise sending an alert to an operator of themovable object. In some embodiments, the one or more flight responsemeasures comprise maintaining a predetermined distance from the target.In some embodiments, the one or more flight response measures comprisetriggering an imaging device to capture images. In some embodiments, theone or more flight response measures comprise a collision avoidancemaneuver. In some embodiments, the collision avoidance maneuvercomprises braking, altering a direction of a moving course of themovable object, stopping one or more propulsion units of the movableobject, deploying one or more airbags, and/or deploying one or moreparachutes. In some embodiments, the one or more passive infraredsensors individually or collectively have a 360° horizontal field ofview around the movable object. In some embodiments, each of the one ormore passive infrared sensors has a detection range of 5 m. In someembodiments, the one or more processors are configured to differentiatethe target from different types of targets based on the heat signal. Insome embodiments, the target is a human, and the heat signal from thehuman is recognizable as belonging to a human. In some embodiments, theheat signal is about 8-12 um in wavelength.

It shall be understood that different aspects of the invention can beappreciated individually, collectively, or in combination with eachother. Various aspects of the invention described herein may be appliedto any of the particular applications set forth below. Other objects andfeatures of the present invention will become apparent by a review ofthe specification, claims, and appended figures.

INCORPORATION BY REFERENCE

All publications, patents, and patent applications mentioned in thisspecification are herein incorporated by reference to the same extent asif each individual publication, patent, or patent application wasspecifically and individually indicated to be incorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth with particularity inthe appended claims. A better understanding of the features andadvantages of the present invention will be obtained by reference to thefollowing detailed description that sets forth illustrative embodiments,in which the principles of the invention are utilized, and theaccompanying drawings of which:

FIG. 1 illustrates a method of recognizing an obstacle, in accordancewith embodiments.

FIG. 2 illustrates a top down view of a device for human collisionavoidance of UAV, in accordance with embodiments.

FIG. 3 illustrates a UAV with a plurality of PIR sensors detecting anobstacle, in accordance with embodiments.

FIG. 4 illustrates a method for responding to a target, in accordancewith embodiments.

FIG. 5 illustrates a movable object tracking a target, in accordancewith embodiments.

FIG. 6 provides a block diagram of a device for human collisionavoidance of UAV, in accordance with embodiments.

FIG. 7 illustrates a UAV operating in an outdoor environment, inaccordance with embodiments.

FIG. 8 illustrates a UAV operating in an indoor environment, inaccordance with embodiments.

FIG. 9 illustrates an unmanned aerial vehicle (UAV), in accordance withembodiments.

FIG. 10 illustrates a movable object including a carrier and a payload,in accordance with embodiments.

FIG. 11 illustrates a schematic illustration by way of block diagram ofa system for controlling a movable object, in accordance withembodiments.

DETAILED DESCRIPTION

Methods, apparatuses, and systems of recognizing or detecting anobstacle are provided. In some embodiments, the obstacle may refer to atarget, and methods, apparatuses, and systems described with respect torecognition or detection of obstacles will be understood to be equallyapplicable to recognition or detection of targets. The recognition ordetection of obstacles may be via sensors located on board a movableobject. As used herein, a movable object may refer to any object thatmay be moved as described elsewhere. For example, a movable object maybe a mobile phone, a watch, an unmanned aerial vehicle (UAV), a car, aboat, a computer, a PDA, a tablet, etc. While many embodiments hereinare described with reference to UAVs, it shall be understood that thereference is non-limiting, and that the embodiments are equallyapplicable to any movable object. In some instances, a distance ofmovable objects to obstacles may be determined with aid of one or moresensors. For example, one or more passive infrared (PIR) sensors mayreceive infrared radiation (e.g., heat signal or pattern) emitted orreflected from an obstacle (e.g., object or subject). The obstacle maybe detected, recognized or identified based on the infrared radiationpattern and a distance from the movable object to the obstacle may becalculated based on data from the PIR sensor. Subsequently, a flightresponse measure may be determined and implemented based on the distancecalculated.

FIG. 1 illustrates a method 100 of recognizing an obstacle, inaccordance with embodiments. Obstacle as used herein may refer to anyexternal factor capable of obscuring a path of a movable object (e.g.,UAV) or capable of coming into contact with the movable object. Theobstacle may comprise any objects or subjects. For example, the obstaclemay be an object that is stationary (e.g., building, boulder, parkedcar, etc), mobile (e.g., moving car, plane, etc), artificial (e.g.,signpost, boat, etc), and/or natural (e.g., tree, waterfall, etc). Forexample, the obstacle may be a subject such as an animal (e.g., cat,dog, horse, etc) or a human being.

In step 102, one or more heat signals from the obstacle may be received.Infrared radiation may also be referred to as a heat signal herein. Allobstacles may have an infrared radiation energy associated with it. Allobstacles may emit or reflect infrared radiation. In some instances,infrared radiation emitted or reflected from an obstacle may be receivedwith aid of infrared (IR) sensors such as PIR sensors. PIR sensors maycomprise pyroelectric materials. PIR sensors (e.g., the pyroelectricmaterials) may detect infrared radiation or changes in an amount ofinfrared radiation (e.g., emitted or reflected from the obstacle)impinging upon it. The pyroelectric materials may generate energy whenexposed to the infrared radiation. For example, the PIR sensors maygenerate energy in response to infrared radiation 0.2-20 um inwavelength. The pyroelectric materials may generate (e.g., output) a lowfrequency pulse signal (e.g., voltage signal) when exposed to theinfrared radiation (e.g., 0.2-20 um in wavelength). Generation of energy(e.g., a pulse signal) in response to received infrared radiation mayherein be referred to as detection of infrared radiation, detection ofchanges in infrared radiation, and/or detection of the obstacle (e.g.,emitting or reflecting the infrared radiation). The infrared radiationimpinging upon the PIR sensor may depend on the temperature, shape,area, volume, and/or surface characteristics (e.g., texture) ofobstacles in front of the sensor. Obstacles of similar temperature butdifferent surface characteristics may emit different infrared radiationpatterns. Obstacles of similar surface characteristics but differenttemperatures may emit different infrared radiation patterns. Obstaclesof different temperature and different surface characteristics may emitdifferent infrared radiation patterns. Objects of similar surfacecharacteristics (e.g., texture) and similar temperatures may emitsimilar infrared radiation patterns. The output signal (e.g., pulsesignal, voltage signal) by the pyroelectric materials may depend onfactors beyond a presence (e.g., temperature, shape, surfacecharacteristic, etc) of the obstacle. For example, the output signal maydepend on factors such as a distance from an obstacle to the PIR sensor,a direction and speed of the obstacle (e.g., relative to the movableobject), a gait (if applicable) of the obstacle, and/or presence ofmultiple obstacles. In some instances, characteristics of the outputsignal (e.g., magnitude of the output signal) in response to detectionof obstacles may depend on factors such as the temperature, surfacecharacteristics, size, area, volume of the obstacle, distance from theobstacle to the PIR sensor, direction and speed of the obstacle (e.g.,relative to the movable object), and gait of the obstacle (ifapplicable).

PIR sensors may be equal to about or smaller than approximately 5 mm²,10 mm², 20 mm², 30 mm², 40 mm², 50 mm², or 100 mm². PIR sensors may havea field of view. The field of view of the PIR sensor may refer to anextent of the environment that is detectable or sensible by the PIRsensor. The field of view may be described by the relative direction ofthe PIR sensor to the movable object. For example, the field of view maybe oriented vertically, horizontally, upward, downward, side-ways, andthe like relative to the movable object (e.g., a UAV). The field of viewof the PIR sensor may be related to, or depend on the placement of thePIR sensor (e.g., on the UAV), a central axis of the PIR sensor, adetection range of the PIR sensor, and/or an angle of view of the PIRsensor. The PIR sensors may each have a central axis. The central axisof a PIR sensor, which may also be referred to as the “principal axis,”can be a line along which there is some degree of rotational symmetry inthe PIR sensor. The central axis of the PIR sensor may be directedupwards, downwards, to a lateral side, horizontally, vertically, or atany angle relative to the movable object. In some embodiments, thecentral axis of the PIR sensor passes through the center of thecomponents (e.g., pyroelectric sensor) of the PIR sensor.

PIR sensors may have an angle of view. The angle of view of a PIR sensormay refer to the angular extent of a given environment that isdetectable (e.g., sensible) by the PIR sensor. The PIR sensor may havean angle of view equal to about or less than 10°, 15°, 20°, 30°, 45°,60°, 90°, 120°, 150°, 180°, 210°, 240°, 270°, 300°, 330°, or 360°. ThePIR sensor may have a horizontal angle of view equal to about or lessthan 10°, 15°, 20°, 30°, 45°, 60°, 90°, 120°, 150°, 180°, 210°, 240°,270°, 300°, 330°, or 360°. The PIR sensor may have a vertical angle ofview equal to about or less than 10°, 15°, 20°, 30°, 45°, 60°, 90°,120°, 150°, 180°, 210°, 240°, 270°, 300°, 330°, or 360°. Differing PIRsensors may have a same angle of view but a different field of view. PIRsensors may have a detection range in which it can detect obstacles. Thedetection range may be equal to about or less than 1 m, 2 m, 3 m, 4 m, 5m, 10 m, 20 m, or 50 m.

PIR sensors may be enclosed by a housing or embedded within a housing.In some embodiments, PIR sensors may be external to a housing. PIRsensors may comprise optical elements to focus infrared energy onto thesensor surface. In some embodiments, the housing may comprise theoptical elements. The optical elements may comprise lenses and/ormirrors. For example, the lens may be a Fresnel lens. PIR sensors maycomprise filters to limit wavelengths of infrared light reaching thesensor surface. In some embodiments, the housing and/or the opticalelements may comprise the filters. The filters on PIR sensors may limitthe wavelength of infrared radiation impinging upon the sensor surfaceto about 2 um, 4 um, 6 um, 8 um, 10 um, 12 um, 14 um, 20 um, 22 um, 24um, 26 um, 28 um, 30 um. In some embodiments, the filters on PIR sensorsmay limit the wavelength of infrared radiation impinging upon the sensorsurface to a range. For example, using a filter, infrared radiation ofabout 0.2-8 um, 0.2-2 um, 2-4 um, 4-6 um, 6-8 um, 8-14 um, 8-12 um, 8-10um, 10-12 um, 10-14, 14-16 um, 16-18 um, 18-20 um, or 14-20 um mayimpinge upon the sensor surface. In some instances, only infraredradiation (e.g., heat signal) of the wavelength or range of wavelengthsimpinging upon the PIR sensor surface may be detected. In someinstances, only obstacles emitting or reflecting infrared radiation ofthe wavelength or range of wavelengths impinging upon the PIR sensorsurface may be detected. For example, while a PIR sensor may be capableof detecting (e.g., receiving and generating energy or a pulse signal)infrared radiation 0.2-20 um in wavelength, a filter may limit thewavelength of the infrared radiation impinging upon the sensor surface(e.g., to 8-10 um). Using a filter, only infrared radiation of certainwavelengths may be passed to the PIR sensor surface while infraredradiation of other wavelengths are filtered off. Using a filter,specific obstacles may be detected or recognized. For example, using afilter that limits wavelength of infrared radiation able to impinge uponthe PIR sensor surface to a range of about 8-10 um, a human may bedetected or recognized. Using other filters that limit wavelength ofinfrared radiation able to impinge upon the PIR sensor, other obstaclesor objects may be detected or recognized. For example, inanimate objectssuch as cars or buildings emitting infrared radiation of specificwavelength or a range of wavelengths may be detected or recognized.

In some instances, the PIR sensors may be located on board the movableobject. The PIR sensors may be located on board the movable object inany number and in any configuration. PIR sensors may be located on acentral body of the movable object or on a peripheral part of themovable object (e.g., on a carrier coupled to the movable object). PIRsensors of the present disclosure can be situated on any suitableportion of a movable object, such as above, underneath, on the side(s)of, or within a body of the movable object. The PIR sensors may belocated on a center or off-center of the movable object. Some PIRsensors can be mechanically coupled to the movable object such that thespatial disposition and/or motion of the movable object correspond tothe spatial disposition and/or motion of the PIR sensor. The PIR sensorcan be coupled to the movable object via a rigid coupling, such that thePIR does not move relative to the portion of the movable object to whichit is attached. The coupling can be a permanent coupling ornon-permanent (e.g., releasable) coupling. Suitable coupling methods caninclude adhesives, bonding, welding, and/or fasteners (e.g., screws,nails, pins, etc.). Optionally, the PIR sensor can be integrally formedwith a portion of the movable object. Furthermore, the PIR sensor can beelectrically coupled with a portion of the movable object (e.g.,processing unit, control system, data storage, flight controller) so asto enable the data collected by the PIR sensor to be used for variousfunctions of the movable object (e.g., navigation, control, propulsion,collision avoidance maneuver, etc.), such as the embodiments discussedherein. The PIR sensor may be operably coupled with a portion of themovable object (e.g., processing unit, control system, data storage).

One, two, three, four, five, six, seven, eight, nine, ten or more PIRsensors may be located on board the movable object. In some instances, acollective field of view of the one or more PIR sensors on board themovable object may cover a 360° angle around the movable object. In someembodiments, the 360° angle may cover a panoramic view of the horizontalsides of the movable object. In some embodiments, a collective field ofview of the one or more PIR sensors on board the movable object maycover a 360° vertical angle around the movable object. In someembodiments, the fields of view of the one or more PIR sensors may covera 360 angle both horizontally and vertically around the movable object(e.g., covering an entire spherical region around the movable object).In some instances, a collective field of view of the one or more PIRsensors may not cover a 360° angle around the movable object. In someembodiments, a plurality of PIR sensors on board the movable object mayhave identical detection ranges (e.g., 5 m). In some embodiments, aplurality of PIR sensors on board the movable object may have differingdetection ranges. In some embodiments, a plurality of PIR sensors onboard the movable object may filter for different wavelengths (or rangesof wavelengths) of infrared radiation.

FIG. 2 illustrates a top down view of a device for human collisionavoidance of UAV, in accordance with embodiments. The UAV 200 has fourPIR sensors 202, 204, 206, 208 on board the UAV, each with an angle ofview of a. PIR sensors 202, 204, 206, 208 are located on a lateral sideof the UAV. Collectively, PIR sensors 202, 204, 206, 208 have a field ofview that covers a 360° angle around the movable object as shown by theoverlapping fields of view 210, 212, 214, 216. In some instances,different PIR sensors may filter for, receive and detect infraredradiation of same, or similar wavelengths. For example, PIR sensors 202,204, 206, 208 may all filter for, receive and detect infrared radiation10 um (or in a range of 8-12 um) in wavelength. In some instances,different PIR sensors may filter for, receive and detect infraredradiation of different wavelengths. For example, PIR sensors 202, 204may filter for, receive and detect infrared radiation 10 um (or in arange of 8-12 um) in wavelength while PIR sensors 206, 208 may filterfor, receive and detect infrared radiation of other wavelengths. In someinstances, different PIR sensors may have the same detection range. Forexample, PIR sensors 202, 204, 206, 208 may all have an identicaldetection range (e.g., 5 m). In some instances, different PIR sensorsmay have different detection ranges. For example, PIR sensors 202, 204may have a detection range or 10 m while PIR sensors 206, 208 may have adetection range of 5 m.

In step 104, the obstacle may be recognized based on the heat signalthat is received in step 102. Obstacles (e.g., objects and/or subjects)may radiate a heat signal (e.g., infrared radiation) that is specific tothe obstacle. For example, human beings may emit infrared radiation thatis about 8 um, 10 um, 12 um, or in a range of about 8-12 um inwavelength. For example, mammals may emit infrared radiation that isabout 8 um, 10 um, 12 um, or in a range of about 8-12 um in wavelength.In some instances, obstacles that have a temperature of around 37°Celsius may emit infrared radiation that is about 8 um, 10 um, 12 um, orin a range of about 8-12 um in wavelength. In some instances, obstaclessuch as buildings or vehicles may emit infrared radiation of specificwavelengths. In some embodiments, the wavelength of infrared radiationemitted by an obstacle may depend or relate to a temperature, size,volume, shape, and/or surface characteristic of the obstacle. In someinstances, the PIR sensors may be configured to receive specificwavelengths or range of wavelengths of infrared radiation as previouslystated herein (e.g., using filters). The specific wavelengths or rangeof wavelengths of infrared radiation received or detected may correspondto infrared radiation emitted from specific obstacles (e.g., one type ofobstacle such as a human being). Thus, the obstacle may be recognizedbased on the infrared radiation (e.g., heat signal) received in step102.

In some instances, the PIR sensors may be configured to receive ordetect a range of wavelength of infrared radiation that does notcorrespond to infrared radiation emitted from a specific obstacle (e.g.,one type of obstacle such as a human being). In response to receivingheat signals from the obstacle, the PIR sensors may output signalswherein characteristics (e.g., magnitude) of the output signals variesdepending on the type of obstacle heat signals are received from. Forexample, a large output signal by the PIR sensor may correspond to ahuman being. For example, a large output by the PIR sensor maycorrespond to a mammal. For example, a large output by the PIR sensormay correspond to a large mammal (e.g., mammal comparable in size to ahuman or larger). A predetermined list of obstacle types may be stored,e.g., on a local memory on board the PIR sensor and/or the movableobject or off board the movable object but in communication with the PIRsensors. Each of the obstacle type may be correlated to an output signalwith a certain characteristic (e.g., of a certain magnitude) and theobstacle type may be recognized based on the output signal by the PIRsensor. A heat signal from an obstacle may be received by a PIR sensor,the PIR sensor may generate an output signal (e.g., detect theobstacle), the output signal by the PIR sensor may be measured (e.g.,with a microcontroller), and the obstacle may be recognized based on theinfrared radiation (e.g., heat signal) received in step 102.

For example, if the PIR sensor receives an infrared radiation (e.g.,heat signal) of 8 um, 10 um, 12 um, or 8-12 um in wavelength, anobstacle may be detected and recognized as belonging to a human being.In some embodiments, different PIR sensors on board the movable objectmay be configured to detect different obstacles and/or recognize ordistinguish between obstacles. PIR sensors may be configured todistinguish a human from other obstacles. PIR sensors may be configuredto distinguish mammals from other obstacles. PIR sensors may beconfigured to distinguish animals from other obstacles. PIR sensors maybe configured to distinguish between different types of animals (e.g.,cold blooded vs. warm blooded). PIR sensors may be configured todistinguish between different types of mammals (e.g., big vs. small).For example, one or more PIR sensors may be configured to receive ordetect infrared radiation of 8 um, 10 um, 12 um, or in the range of 8-12um in wavelength while one or more other PIR sensors may be configuredto receive or detect infrared radiation of other wavelengths. If PIRsensors configured to receive IR radiation 8 um, 10 um, 12 um (or 8-12um) in wavelength respond (e.g., generate energy, output a pulse signal,etc), obstacles detected may be recognized as humans. If PIR sensorsconfigured to receive IR radiation 8 um, 10 um, 12 um (or 8-12 um) inwavelength do not respond but other PIR sensors do respond, obstaclesdetected may be recognized as non-humans. Step 104 of recognizing theobstacle may include differentiating the obstacle from different typesof obstacles based on the infrared radiation (e.g., heat signal) asdescribed herein (e.g., differentiating between human and non-human).

In some embodiments, step 104 may be optional and the obstacle may notneed to be recognized for subsequent steps. For example, step 106 ofcalculating a distance from the movable object to the obstacle or step108 of determining an appropriate flight response measure may beimplemented as soon as an infrared radiation (e.g., of specificwavelength of range of wavelength) is received or detected.

In step 106, a distance from the movable object to the obstacle may becalculated with aid of one or more processors. A processor may comprisea field-programmable gate array (FPGA), application-specific integratedcircuit (ASIC), application-specific standard product (ASSP), digitalsignal processor (DSP), central processing unit (CPU), graphicsprocessing unit (GPU), vision processing unit (VPU), complexprogrammable logic devices (CPLD), and the like. A processor may be anon-board processor on-board a movable object (e.g., a UAV) or anembedded processor carried by the PIR sensor. A processor may be anoff-board processor separated from the movable object and/or the PIRsensor (e.g., at a ground station, communicating with a UAV and/or PIRsensor). The one or more processors as referred to herein may beindividually or collectively configured to further aid in method 100 asrecited herein. In some instances, a single processor (or CPU) may beconfigured to aid in method 100 for a plurality of PIR sensors. In someinstances, a plurality of processors may be configured to aid in method100 for a single PIR sensor.

The calculated distance may be based on, or depend on data from the PIRsensor. In some instances, the distance from the movable object to theobstacle may be calculated or determined to be the detection range ofthe PIR sensor that detects the obstacle. For example, at the moment aPIR sensor first detects an obstacle, the instantaneous distance fromthe movable object to the obstacle may be determined to be the detectionrange of the PIR sensor. For example, for a PIR sensor having an fieldof view of 360° (or PIR sensors of the same detection range collectivelyhaving a field of view of 360°), the distance from the movable object tothe obstacle at the moment an object is first detected (or recognized)may be substantially equal to the detection range of the PIR sensor. Insome embodiments, the distance from the movable object to the obstacleat the moment an object is detected (or recognized) may be calculated ordetermined (e.g., estimated) to be equal to the detection range of thePIR sensor that detects the obstacle even if the field of view (orcollective fields of view of a plurality of PIR sensors) does not equal360°.

In some instances, the distance from the movable object to the obstaclemay be calculated or determined based on an output signal of the PIRsensor. As previously stated herein, the output signal (e.g., pulsesignal, voltage signal) by the pyroelectric materials may depend onfactors such as a distance from an obstacle to the PIR sensor. Forexample, the strength of the output signal of the PIR sensor mayincrease as a distance from the movable object to the obstacledecreases. In some instances, a relationship between the distance andpeak value of PIR sensor output signal may be calibrated (e.g., inadvance) and a distance corresponding to a peak value of PIR sensoroutput signals may be known. In some instances, one or more processors(e.g., microcontroller) may detect a peak value of an output signal ofthe PIR sensor and calculate or determine a distance based on acalibration curve. Accordingly, a distance from the movable object tothe obstacle at time points subsequent to first detection of theobstacle may be determined. In some instances, other factors (e.g.,speed and direction of the obstacle relative to the movable object) maybe calculated or determined based on an output signal of the PIR sensorunder similar reasoning (e.g., according to output signals of the PIRsensors).

FIG. 3 illustrates a UAV 300 with a plurality of PIR sensors 302, 304,306, 308, and 310 detecting an obstacle 312, in accordance withembodiments. PIR sensors 302, 304, 306, 308 are located on a lateralside of the UAV while PIR sensor 310 is located on an upper surface(e.g., sky-facing surface) of the UAV. PIR sensor 310 has a horizontalangle of view of 360 degrees and a range of 3 m defined by 314. PIRsensors 302, 304, 306, 308 each has a horizontal angle of view of αdegrees and a detection range of 5 m. PIR sensors 302, 304, 306, 308collectively have a field of view that covers a 360° angle around themovable object 300 as shown by the overlapping fields of view. In someinstances, PIR sensors of a certain detection range may collectively nothave a field of view that covers a 360° angle around the movable object,and there may be blind spots in which the PIR sensors cannot detectobstacles at a certain distance. PIR sensor 310 may be used to detectobstacles roughly at a distance of 3 m or less to the movable object.PIR sensors 302, 304, 306, 308 may be used to detect obstacles roughlyat a distance of 5 m or less to the movable object. PIR sensors 302,304, 306, 308, 310 may all filter for and receive infrared radiation ofcertain wavelengths. For example, the PIR sensors may receive IRradiation approximately 10 um (or in a range of 8-12 um) in wavelengthand generate energy or an output signal. When an obstacle is firstdetected by PIR sensor 302, 304, 306, or 308, the distance from themovable object to the obstacle may be calculated or determined to be 5m. When an obstacle is first detected by PIR sensor 310, the distancefrom the movable object to the obstacle may be calculated or determinedto be 3 m. The distance from the movable object 300 to obstacle 312 maybe calculated by detecting a peak value of output signal of the PIRsensor (e.g., using a microcontroller) and comparing the peak value to acalibration curve (e.g., calibration curve showing the relationshipbetween the distance and peak value of PIR sensor output signal). Forexample, using a calibration curve, the distance may be calculated ordetermined to be anywhere in a range of 0 m to 5 m. The distance may becalculated to an accuracy of about or within 0.1 m, 0.2 m, 0.3 m, 0.4 m,0.5 m, 1 m, or 2 m of the actual distance.

In some embodiments, a relative position of the movable object to theobstacle may be determined, e.g., based on a direction of mounting ofthe PIR sensor (e.g., field of view, direction of a central axis of thePIR sensor) and a distance to the obstacle. One or more processors maydetermine which PIR sensor is generating an output signal and centralaxis of each of the PIR sensors, or a direction which the PIR sensorsare mounted may be predetermined or stored in a memory operably coupledto one or more processors. A distance from the obstacle to the movableobject may be calculated or determined as previously described herein.For example, FIG. 3 shows obstacle 312 within a field of view of PIRsensor 308. PIR sensor 308 may generate an output signal because theobstacle 312 is in its field of view while PIR sensors 302, 304, 306,310 may generate no output signal. A distance from the obstacle 312 tomovable object 300 may be calculated to be 5 m. A mounting direction ora direction of the central axis 316 of PIR sensor may be stored in amemory unit. Based on the distance and the central axis, a position ofthe obstacle relative to the movable object may be estimated to be atlocation 318. Using additional PIR sensors and/or PIR sensors with asmaller angle of view, a more accurate and precise determination of theposition of obstacles may be made.

In some embodiments, step 106 may be optional and a distance from themovable object to the obstacle may not need to be determined orcalculated for subsequent steps. For example, step 108 of determining anappropriate flight response measure may be implemented as soon as anobstacle is detected and/or recognized.

In step 108, whether to effect (e.g., implement, perform) a flightresponse measure may be determined. In some embodiments, the flightresponse measure may be a collision avoidance maneuver. Determiningwhether to effect a flight response measure may comprise determining anappropriate flight response measure (e.g., collision avoidancemaneuver). The appropriate flight response measure and/or whether toeffect a flight response measure (e.g., collision avoidance maneuver)may be determined and adopted with aid of a flight controller. Thedetermination may be made on-board the movable object (e.g., using oneor more processors on board the movable object). The determination maybe made off-board the movable (e.g., using one or more processorsoperably coupled to the movable object). The determination may be madeautomatically (e.g., with aid of the one or more processors, with aid ofthe flight controller). The determination may be made manually orsemi-automatically (e.g., with some user input). The determination maybe based on one or more factors such as a distance from the movableobject to the obstacle (e.g., calculated in step 106), direction ofmovement of the movable object, speed of the movable object, directionof movement of the obstacle, speed of the obstacle, relative position ofthe obstacle to the movable object, obstacle type (e.g., recognized instep 104), capabilities of the movable object (e.g., acceleration ordeceleration capabilities), size and type of the movable object, sizeand type of the obstacle, and/or an altitude of the movable object. Insome instances, the factors (e.g., distance from the movable object tothe obstacle) may be of data acquired using PIR sensors as mentionedherein. In some instances, the factors may be of data acquired usingother sensors (e.g., IMU sensors measuring a speed of the movableobject).

In some embodiments, the flight response measure (e.g., collisionavoidance maneuver) may comprise decreasing or limiting a speed of themovable object. In some embodiments, the flight response measure (e.g.,collision avoidance maneuver) may comprise stopping one or morepropulsion units of the movable object. In some embodiments, the flightresponse measure (e.g., collision avoidance maneuver) may comprisebraking or stopping movement of the movable object. In some embodiments,the flight response measure (e.g., collision avoidance maneuver) maycomprise altering a direction of the moving course of the movableobject. In some embodiments, the flight response measure (e.g.,collision avoidance maneuver) may comprise increasing or decreasing analtitude of the movable object. In some embodiments, the flight responsemeasure may comprise sending an alert (e.g., auditory, visual, tactilewarning signal) to an operator of the movable object. In someembodiments, the flight response measure may comprise sending an alert(e.g., auditory, visual, tactile warning signal) to the detectedobstacle or target. In some embodiments, the flight response measure maycomprise triggering one or more imaging devices to capture images (e.g.,of the obstacle and/or environment around the movable object). In someembodiments, the flight response measure may comprise tracking a target(e.g., the obstacle). In some embodiments, tracking a target maycomprise following the target and/or capturing one or more images of thetarget. The target may be followed at a specified distance. In someembodiments, the flight response measure (e.g., collision avoidancemaneuver) may comprise deploying one or more airbags on board themovable object and/or deploying one or more parachutes on board themovable object.

In some instances, one or more flight response measures may beperformed, effected or implemented depending on the aforementionedfactors such as the distance. For example, different flight responsemeasures may be effected depending on one or more threshold distances.For example, there may be one threshold distance (e.g., 5 m). A distancefrom the movable object to the obstacle or the target may be determined,and compared to the threshold distance. If an obstacle is detected to bewithin the threshold distance, a flight response measure may be effected(e.g., braking). Outside of the threshold distance, a second flightresponse measure may be effected (e.g., sending a warning signal) or noflight response measure may be effected. In some instances, the one ormore flight response measures to be effected may depend on a pluralityof factors (two or more factors) such as a distance and a speed of themovable object (e.g., movable object) relative to the obstacle.

For example, there may be two threshold distances (e.g., 2 m and 5 m). Adistance from the movable object to the obstacle or the target may bedetermined, and compared to the threshold distances. If an obstacle isdetected to be within the first threshold distance (e.g., 2 m), a firstflight response measure may be implemented, such as braking or stoppingone or more propulsion units of the movable object. If an obstacle isdetected to be outside the first threshold distance (e.g., 2 m) butwithin the second threshold distance (e.g., 5 m), a second flightresponse measure may be implemented, such as altering a direction ofmovement of the movable object or decreasing a velocity of the movableobject. Outside of the second threshold distance, a third flightresponse measure may be effected (e.g., sending a warning signal) or noflight response measure may be effected. In some instances, the one ormore flight response measures to be effected may depend on a pluralityof factors (two or more factors) such as a distance and a speed of themovable object (e.g., movable object) relative to the obstacle.

Method 100 may further comprise effecting the appropriate flightresponse strategy (e.g., collision avoidance maneuver) when adetermination is made in step 108.

FIG. 4 illustrates a method 400 for responding to a target, inaccordance with embodiments. In step 402, one or more heat signals froma target may be received (e.g., by PIR sensors). The heat signals may befor example, infrared radiation ranging in wavelength from 0.2 to 20 um,as described herein. The target may comprise the obstacles as referredherein, including objects and subjects such as human beings, cars,buildings, animals, etc. In step 404 the target may be detected orrecognized, previously as described herein. For example, the PIR sensormay generate energy or a pulse signal in response to the receivedinfrared radiation (e.g., 0.2-20 um in wavelength) and accordinglydetect the obstacle. For example, the PIR sensors may comprise filtersthat selectively allow infrared radiation of certain wavelengths (e.g.,8-10 um) to reach the PIR sensor surface. The PIR sensor may generateenergy or a pulse signal in response to the received infrared radiationand detect the obstacle. The detected obstacle may be recognized as atype of obstacle (e.g., human) due to the filter limiting the range ofwavelengths impinging upon the PIR sensor surface. The detected obstaclemay be recognized as a type of obstacle via other methods as previouslydescribe herein (e.g., based on an output signal of the PIR sensor). Instep 406 one or more flight response measures may be performed based onthe detected or recognized target. The flight response measures may beas previously described herein (e.g., tracking a target, sending analert or a warning signal, altering a flight path, etc).

In some embodiments, the flight response measure may comprise tracking atarget. Tracking the target may comprise following or trailing thetarget and/or capturing images of the target using one or more imagingdevices. In some instances, the target may be followed or trailed at adesired distance. The desired distance may be predetermined and/or maybe updated with user input in real time (e.g., while the target is beingtracked). In some instances, the flight response measure may comprisemaintaining a desired distance (e.g., a trailing distance) from thetarget (e.g., obstacle). For example, a target may be detected and adistance may be calculated, as previously described herein. A desiredtrailing distance (e.g., threshold distance) may be determined (e.g., 5m, predetermined) and a peak output signal of the PIR sensors may bemeasured and monitored. A PIR sensor peak output signal corresponding tothe trailing distance may be determined according to a calibrationcurve. If the peak output signal of the PIR sensor is greater than apeak output signal corresponding to the trailing distance, the movableobject may move away from the obstacle (e.g., based on the relativeposition and/or direction of the movable object from the obstacle). Ifthe peak output signal of the PIR sensor is less than a peak outputsignal corresponding to the trailing distance, the movable object maymove towards the obstacle (e.g., based on the relative position and/ordirection of the movable object from the obstacle). Accordingly, a“follow” function may be achieved wherein the movable object follows ortrails a target (e.g., an obstacle, a human being) while maintaining adesired trailing distance.

The desired trailing distance may be updated, e.g., using user input.For example, the user may specify the movable object to follow thetarget at a distance of about 1 m, 2 m, 3 m, 4 m, 5 m, 6 m, 7 m, 8 m, 9m, 10 m, or more. The target may be tracked or followed at a specifiedposition relative to the target (e.g., at a specified altitude relativeto the target, at a specified vertical distance relative to the target,at a specified horizontal distance relative to the target). For example,the movable object may track (e.g., follow) the target at a horizontalplane directly above the target while maintaining the desired distance.For example, the movable object may track the target at on the samehorizontal plane as the target while maintaining the desired distance.For example, the movable object may track the target while maintainingan equidistant horizontal and vertical distance from the target. In someembodiments, a desired position of the movable object relative to thetarget may be updated (e.g., in real time). For example, a movableobject may be configured to track the target at a distance of about 5 mequidistant horizontally and vertically from the target. A user maysubsequently specify the movable object to track the target directlyabove the target while maintaining the trailing distance. In someembodiments, the movable object may track the target while sending analert (e.g., visual, auditory, or tactile warning signal) to thedetected target, obstacle, and/or operator of the movable object. Insome embodiments, one or more images of the target may be captured. Themovable object may track the target while capturing images of the targetor obstacle. In some embodiments, the movable object may capture imagesof the target while following (e.g., trailing) the target.

FIG. 5 illustrates a movable object tracking a target, in accordancewith embodiments. The movable object may receive a heat signal (e.g.,infrared radiation) from the human being (e.g., the target) and trackthe target (e.g., capture images of the target or follow the target). Ahuman being is shown moving from position 503 to position 507 over atime period. A movable object receives a heat signal from the target.The movable object may detect the target, recognize the target based onthe received heat signal (e.g., infrared radiation), and determine adistance 504 from the movable object to the target. The movable objectmay follow or trail the target from position 502 to position 506 as thetarget moves from position 503 to position 505. The distance from themovable object to the target 504 at the earlier time point and thedistance from the movable object to the target 508 at the later timepoint may be identical. The movable object may maintain the distanceand/or a relative position to the target while it tracks the target(e.g., follows the target and/or captures images of the target). Themovable object may vary or change the distance and/or relative positionto the target while it tracks the target (e.g., with external input).

FIG. 6 illustrates a block diagram for human collision avoidance ofUAVs, in accordance with embodiments. A human may emit a thermal signal(e.g., infrared radiation). The thermal signal may be emitted at awavelength as previously described herein (e.g., 10 um, 8-12 um, etc).The thermal signal may be focused onto a sensor surface, for example,using optical elements such as a Fresnel lens. In some embodiments, theoptical elements may amplify the infrared radiation emitted from thehuman and focus the IR radiation on a PIR (e.g., pyroelectric IR) sensorsurface. The PIR sensor may receive the thermal signal after the thermalsignal has passed through an optical element such as the Fresnel lens.The pyroelectric IR sensor may receive the IR radiation and in response,output energy, or a low frequency pulse signal. The pulse signal may beprocessed by an amplifier-filter circuit and then be transferred to adigital signal by an analog-to-digital (AD) converter. In someembodiments, the AD converter may be a comparator. The digital signalmay be output to one or more processors such as a computer processingunit (CPU) which may determine whether an obstacle is around, whetherthe obstacle is a human, and a distance of the obstacle from the movableobject. This information may be informed to a flight controller whichmay adopt appropriate flight response strategies in order to implementhuman collision avoidance. In some embodiments, the flight controllermay comprise the one or more processors (e.g., CPU). For example, if anobstacle is detected and recognized as a human, a distance may becalculated. In some embodiments, if the distance is calculated to be 10m or more, the collision avoidance maneuver may be to increase thealtitude of the movable object, to decelerate the movable object, or toalter a direction of the moving course of the UAV. In some embodiments,if the distance is calculated to be 5 m or less, the collision avoidancemaneuver may be to decelerate (e.g., brake) or to stop the one or morepropulsion units of the movable object.

The apparatus, methods, and computer readable mediums disclosed hereinmay offer improved operational capabilities (e.g., collision avoidancemaneuver) for movable objects. FIG. 7 illustrates a UAV 702 operating inan outdoor environment 700, in accordance with embodiments. The outdoorenvironment 700 may be an urban, suburban, or rural setting, or anyother environment that is not at least partially within a building. TheUAV 702 may be operated relatively close to the ground 704 (e.g., lowaltitude) or relatively far from the ground 704 (e.g., high altitude).For example, a UAV 702 operating less than or equal to approximately 10m from the ground may be considered to be at low altitude, while a UAV702 operating at greater than or equal to approximately 10 m from theground may be considered to be at high altitude.

In some embodiments, the outdoor environment 700 includes one or moreobjects 708 a-d. Some objects may be situated on the ground 704 (e.g.,objects 708 a, 708 d), such as buildings, ground vehicles (e.g., cars,motorcycles, trucks, bicycles), human beings, animals, plants (e.g.,trees, bushes), and other manmade or natural structures. Some objectsmay be in contact with and/or supported by the ground 704, water,manmade structures, or natural structures. Alternatively, some objectsmay be wholly located in the air 706 (e.g., objects 708 b, 708 c) andmay be mobile (e.g., aerial vehicles, airplanes, helicopters, hot airballoons, UAVs, or birds). Aerial objects may not be supported by theground 704, or by water, or by any natural or manmade structures. Anobstacle located on the ground 704 may include portions that extendsubstantially into the air 706 (e.g., tall structures such as towers,skyscrapers, lamp posts, radio towers, power lines, trees, etc.).

In some instances, sensors such as Lidar sensors may be utilized forobstacle avoidance in an outdoor environment 700. For example, a 3DLidar module may be provided on a UAV. A laser beam may be emitted and a3-dimensional coordinate of a scanned spot may be obtained using acamera. A distance from the UAV to a nearby obstacle may be furtherdetermined and obstacle avoidance may be achieved. In some embodiments,lighting conditions may deteriorate performance of the Lidar sensors inachieving obstacle avoidance. For example, 3D Lidar may be interfered bysunlight. PIR sensors may be utilized as an alternative or supplement tothe other sensors or methods (e.g., Lidar sensors) for obstacleavoidance in an outdoor environment. For example, PIR sensors may beutilized in areas in which Lidar sensors do not work. PIR sensors may beable to detect and avoid obstacles regardless of lighting conditions inan environment (e.g., outdoor environment).

In some instances, one or more PIR sensors coupled to the UAV 702 maydetect obstacles in an outdoor environment 700. In some instances, thePIR sensors may be configured to selectively detect and recognizeobstacles, such as human obstacles. For example, the PIR sensors may beable to differentiate certain obstacles from different types ofobstacles based on a heat signal (e.g., infrared radiation) emitted bythe obstacles. PIR sensors or one or more processors coupled to the PIRsensor may further calculate or determine a distance from the UAV to theobstacle based on data from the PIR sensor and determine whether toeffect a collision avoidance maneuver as previously described herein.

FIG. 8 illustrates a UAV 852 operating in an indoor environment 850, inaccordance with embodiments. The indoor environment 850 is within theinterior of a building 854 having a floor 856, one or more walls 858,and/or a ceiling or roof 860. Exemplary buildings include residential,commercial, or industrial buildings such as houses, apartments, offices,manufacturing facilities, storage facilities, and so on. The interior ofthe building 854 may be completely enclosed by the floor 856, walls 858,and ceiling 860 such that the UAV 852 is constrained to the interiorspace. Conversely, at least one of the floor 856, walls 858, or ceiling860 may be absent, thereby enabling the UAV 852 to fly from inside tooutside, or vice-versa. Alternatively or in combination, one or moreapertures 864 may be formed in the floor 856, walls 858, or ceiling 860(e.g., a door, window, skylight).

Similar to the outdoor environment 700, the indoor environment 850 caninclude one or more objects 862 a-d. Some objects may be situated on thefloor 856 (e.g., obstacle 862 a), such as furniture, appliances, humanbeings, animals, plants, and other manmade or natural objects.Conversely, some objects may be located in the air (e.g., object 862 b),such as birds or other UAVs. Some obstacles in the indoor environment850 can be supported by other structures or objects. Objects may also beattached to the ceiling 860 (e.g., obstacle 862 c), such as lightfixtures, ceiling fans, beams, or other ceiling-mounted appliances orstructures. In some embodiments, objects may be attached to the walls858 (e.g., obstacle 862 d), such as light fixtures, shelves, cabinets,and other wall-mounted appliances or structures. Notably, the structuralcomponents of the building 854 can also be considered to be objects,including the floor 856, walls 858, and ceiling 860.

The objects described herein may be substantially stationary (e.g.,buildings, plants, structures) or substantially mobile (e.g., humanbeings, animals, vehicles, or other objects capable of movement). Someobjects may include a combination of stationary and mobile components(e.g., a windmill). Mobile objects or obstacle components may moveaccording to a predetermined or predictable path or pattern. Forexample, the movement of a car may be relatively predictable (e.g.,according to the shape of the road). Alternatively, some mobile objectsor object components may move along random or otherwise unpredictabletrajectories. For example, a living being such as an animal may move ina relatively unpredictable manner.

In some instances, sensors such as vision sensors may be utilized forobstacle avoidance within an indoor environment 850. For example, astereo vision module may be provided on a UAV. Images may be capturedfrom two cameras, and depth information may be acquired via visual imageprocessing methods. Subsequently, a distance from the UAV to a nearbyobstacle may be determined and obstacle avoidance may be achieved. Insome embodiments, lighting conditions may deteriorate performance of thevision sensors in achieving obstacle avoidance. For example, stereovision may not work without visible light and may be disabled in dim ordark lighting conditions. PIR sensors may be utilized as an alternativeor supplement to the other sensors or methods (e.g., vision sensors) forobstacle avoidance in an indoor environment. For example, PIR sensorsmay be utilized in areas in which vision sensors do not work. PIRsensors may be able to detect and avoid obstacles regardless of lightingconditions in an environment (e.g., indoor environment).

In some instances, one or more PIR sensors coupled to the UAV 852 maydetect obstacles in an indoor environment 850. In some instances, thePIR sensors may be configured to selectively detect and recognizeobstacles, such as human obstacles. For example, the PIR sensors may beable to differentiate certain obstacles from different types ofobstacles based on a heat signal (e.g., infrared radiation) emitted bythe obstacles. PIR sensors or one or more processors coupled to the PIRsensor may further calculate or determine a distance from the UAV to theobstacle based on data from the PIR sensor and determine whether toeffect a collision avoidance maneuver as previously described herein.

The embodiments provided herein may enable use of low cost pyroelectricinfrared sensors for collision avoidance (e.g., human collisionavoidance) for movable objects such as UAVs. The use of PIR sensors forcollision avoidance may be used alternatively to, or in conjunction withother sensors such as vision sensors (e.g., stereo vision sensor) orLidar sensors (e.g., 3D Lidar sensor) to avoid contact of UAVs withobstacles such as humans. The PIR sensors as used herein may be utilizedboth in indoor and outdoor environments for obstacle avoidance and mayrequire relatively simple algorithms and may be used with a variety ofprocessors (e.g., cheap processors, processors with low processingpower, etc) in order to effect human collision avoidance. The PIRsensors may comprise filters to selectively detect or recognizeobstacles and appropriate flight response measures may be implemented(e.g., only in regards to select or predetermined types of obstaclessuch as animals or humans).

The embodiments provided herein can be applied to various types of UAVs.For instance, the UAV may be a small-scale UAV that weighs no more than10 kg and/or has a maximum dimension of no more than 1.5 m. In someembodiments, the UAV may be a rotorcraft, such as a multi-rotor aircraftthat is propelled to move through the air by a plurality of propellers(e.g., a quadcopter). Additional examples of UAVs and other movableobjects suitable for use with the embodiments presented herein aredescribed in further detail below.

The UAVs described herein can be operated completely autonomously (e.g.,by a suitable computing system such as an onboard controller),semi-autonomously, or manually (e.g., by a human user). The UAV canreceive commands from a suitable entity (e.g., human user or autonomouscontrol system) and respond to such commands by performing one or moreactions. For example, the UAV can be controlled to take off from theground, move within the air (e.g., with up to three degrees of freedomin translation and up to three degrees of freedom in rotation), move totarget location or to a sequence of target locations, hover within theair, land on the ground, and so on. As another example, the UAV can becontrolled to move at a specified velocity and/or acceleration (e.g.,with up to three degrees of freedom in translation and up to threedegrees of freedom in rotation) or along a specified movement path.Furthermore, the commands can be used to control one or more UAVcomponents, such as the components described herein (e.g., sensors,actuators, propulsion units, payload, etc.). For instance, some commandscan be used to control the position, orientation, and/or operation of aUAV payload such as a camera. Optionally, the UAV can be configured tooperate in accordance with one or more predetermined operating rules.The operating rules may be used to control any suitable aspect of theUAV, such as the position (e.g., latitude, longitude, altitude),orientation (e.g., roll, pitch yaw), velocity (e.g., translationaland/or angular), and/or acceleration (e.g., translational and/orangular) of the UAV. For instance, the operating rules can be designedsuch that the UAV is not permitted to fly beyond a threshold height,e.g., the UAV can be configured to fly at a height of no more than 400 mfrom the ground. In some embodiments, the operating rules can be adaptedto provide automated mechanisms for improving UAV safety and preventingsafety incidents. For example, the UAV can be configured to detect arestricted flight region (e.g., an airport) and not fly within apredetermined distance of the restricted flight region, thereby avertingpotential collisions with aircraft and other obstacles.

The systems, devices, and methods described herein can be applied to awide variety of movable objects. As previously mentioned, anydescription herein of a UAV may apply to and be used for any movableobject. A movable object of the present invention can be configured tomove within any suitable environment, such as in air (e.g., a fixed-wingaircraft, a rotary-wing aircraft, or an aircraft having neither fixedwings nor rotary wings), in water (e.g., a ship or a submarine), onground (e.g., a motor vehicle, such as a car, truck, bus, van,motorcycle; a movable structure or frame such as a stick, fishing pole;or a train), under the ground (e.g., a subway), in space (e.g., aspaceplane, a satellite, or a probe), or any combination of theseenvironments. The movable object can be a vehicle, such as a vehicledescribed elsewhere herein. The movable object may be a self-propelledunmanned vehicle that does not require human input. In some embodiments,the movable object can be mounted on a living subject, such as a humanor an animal. Suitable animals can include avians, canines, felines,equines, bovines, ovines, porcines, delphines, rodents, or insects. Insome embodiments, the movable object may be carried.

The movable object may be capable of moving freely within theenvironment with respect to six degrees of freedom (e.g., three degreesof freedom in translation and three degrees of freedom in rotation).Alternatively, the movement of the movable object can be constrainedwith respect to one or more degrees of freedom, such as by apredetermined path, track, or orientation. The movement can be actuatedby any suitable actuation mechanism, such as an engine or a motor. Theactuation mechanism of the movable object can be powered by any suitableenergy source, such as electrical energy, magnetic energy, solar energy,wind energy, gravitational energy, chemical energy, nuclear energy, orany suitable combination thereof. The movable object may beself-propelled via a propulsion system, as described elsewhere herein.The propulsion system may optionally run on an energy source, such aselectrical energy, magnetic energy, solar energy, wind energy,gravitational energy, chemical energy, nuclear energy, or any suitablecombination thereof. Alternatively, the movable object may be carried bya living being.

In some instances, the movable object can be a vehicle. Suitablevehicles may include water vehicles, aerial vehicles, space vehicles, orground vehicles. For example, aerial vehicles may be fixed-wing aircraft(e.g., airplane, gliders), rotary-wing aircraft (e.g., helicopters,rotorcraft), aircraft having both fixed wings and rotary wings, oraircraft having neither (e.g., blimps, hot air balloons). A vehicle canbe self-propelled, such as self-propelled through the air, on or inwater, in space, or on or under the ground. A self-propelled vehicle canutilize a propulsion system, such as a propulsion system including oneor more engines, motors, wheels, axles, magnets, rotors, propellers,blades, nozzles, or any suitable combination thereof. In some instances,the propulsion system can be used to enable the movable object to takeoff from a surface, land on a surface, maintain its current positionand/or orientation (e.g., hover), change orientation, and/or changeposition.

The movable object can be controlled remotely by a user or controlledlocally by an occupant within or on the movable object. In someembodiments, the movable object is an unmanned movable object, such as aUAV. An unmanned movable object, such as a UAV, may not have an occupantonboard the movable object. The movable object can be controlled by ahuman or an autonomous control system (e.g., a computer control system),or any suitable combination thereof. The movable object can be anautonomous or semi-autonomous robot, such as a robot configured with anartificial intelligence.

The movable object can have any suitable size and/or dimensions. In someembodiments, the movable object may be of a size and/or dimensions tohave a human occupant within or on the vehicle. Alternatively, themovable object may be of size and/or dimensions smaller than thatcapable of having a human occupant within or on the vehicle. The movableobject may be of a size and/or dimensions suitable for being lifted orcarried by a human. Alternatively, the movable object may be larger thana size and/or dimensions suitable for being lifted or carried by ahuman. In some instances, the movable object may have a maximumdimension (e.g., length, width, height, diameter, diagonal) of less thanor equal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m, or 10 m. Themaximum dimension may be greater than or equal to about: 2 cm, 5 cm, 10cm, 50 cm, 1 m, 2 m, 5 m, or 10 m. For example, the distance betweenshafts of opposite rotors of the movable object may be less than orequal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m, or 10 m.Alternatively, the distance between shafts of opposite rotors may begreater than or equal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m,or 10 m.

In some embodiments, the movable object may have a volume of less than100 cm×100 cm×100 cm, less than 50 cm×50 cm×30 cm, or less than 5 cm×5cm×3 cm. The total volume of the movable object may be less than orequal to about: 1 cm³, 2 cm³, 5 cm³, 10 cm³, 20 cm³, 30 cm³, 40 cm³, 50cm³, 60 cm³, 70 cm³, 80 cm³, 90 cm³, 100 cm³, 150 cm³, 200 cm³, 300 cm³,500 cm, 750 cm³, 1000 cm³, 5000 cm³, 10,000 cm³, 100,000 cm³, 1 m³, or10 m³. Conversely, the total volume of the movable object may be greaterthan or equal to about: 1 cm³, 2 cm³, 5 cm³, 10 cm³, 20 cm³, 30 cm³, 40cm³, 50 cm³, 60 cm³, 70 cm³, 80 cm³, 90 cm³, 100 cm³, 150 cm³, 200 cm³,300 cm, 500 cm³, 750 cm³, 1000 cm³, 5000 cm³, 10,000 cm³, 100,000 cm³, 1m³, or 10 m³.

In some embodiments, the movable object may have a footprint (which mayrefer to the lateral cross-sectional area encompassed by the movableobject) less than or equal to about: 32,000 cm², 20,000 cm², 10,000 cm²,1,000 cm², 500 cm², 100 cm², 50 cm², 10 cm², or 5 cm². Conversely, thefootprint may be greater than or equal to about: 32,000 cm², 20,000 cm²,10,000 cm², 1,000 cm², 500 cm², 100 cm², 50 cm², 10 cm², or 5 cm².

In some instances, the movable object may weigh no more than 1000 kg.The weight of the movable object may be less than or equal to about:1000 kg, 750 kg, 500 kg, 200 kg, 150 kg, 100 kg, 80 kg, 70 kg, 60 kg, 50kg, 45 kg, 40 kg, 35 kg, 30 kg, 25 kg, 20 kg, 15 kg, 12 kg, 10 kg, 9 kg,8 kg, 7 kg, 6 kg, 5 kg, 4 kg, 3 kg, 2 kg, 1 kg, 0.5 kg, 0.1 kg, 0.05 kg,or 0.01 kg. Conversely, the weight may be greater than or equal toabout: 1000 kg, 750 kg, 500 kg, 200 kg, 150 kg, 100 kg, 80 kg, 70 kg, 60kg, 50 kg, 45 kg, 40 kg, 35 kg, 30 kg, 25 kg, 20 kg, 15 kg, 12 kg, 10kg, 9 kg, 8 kg, 7 kg, 6 kg, 5 kg, 4 kg, 3 kg, 2 kg, 1 kg, 0.5 kg, 0.1kg, 0.05 kg, or 0.01 kg.

In some embodiments, a movable object may be small relative to a loadcarried by the movable object. The load may include a payload and/or acarrier, as described in further detail below. In some examples, a ratioof a movable object weight to a load weight may be greater than, lessthan, or equal to about 1:1. In some instances, a ratio of a movableobject weight to a load weight may be greater than, less than, or equalto about 1:1. Optionally, a ratio of a carrier weight to a load weightmay be greater than, less than, or equal to about 1:1. When desired, theratio of an movable object weight to a load weight may be less than orequal to: 1:2, 1:3, 1:4, 1:5, 1:10, or even less. Conversely, the ratioof a movable object weight to a load weight can also be greater than orequal to: 2:1, 3:1, 4:1, 5:1, 10:1, or even greater.

In some embodiments, the movable object may have low energy consumption.For example, the movable object may use less than about: 5 W/h, 4 W/h, 3W/h, 2 W/h, 1 W/h, or less. In some instances, a carrier of the movableobject may have low energy consumption. For example, the carrier may useless than about: 5 W/h, 4 W/h, 3 W/h, 2 W/h, 1 W/h, or less. Optionally,a payload of the movable object may have low energy consumption, such asless than about: 5 W/h, 4 W/h, 3 W/h, 2 W/h, 1 W/h, or less.

FIG. 9 illustrates an unmanned aerial vehicle (UAV) 900, in accordancewith embodiments. The UAV may be an example of a movable object asdescribed herein. The UAV 900 can include a propulsion system havingfour rotors 902, 904, 906, and 908. Any number of rotors may be provided(e.g., one, two, three, four, five, six, seven, eight, or more). Therotors, rotor assemblies, or other propulsion systems of the unmannedaerial vehicle may enable the unmanned aerial vehicle to hover/maintainposition, change orientation, and/or change location. The distancebetween shafts of opposite rotors can be any suitable length 910. Forexample, the length 910 can be less than or equal to 2 m, or less thanequal to 5 m. In some embodiments, the length 910 can be within a rangefrom 40 cm to 1 m, from 10 cm to 2 m, or from 5 cm to 5 m. Anydescription herein of a UAV may apply to a movable object, such as amovable object of a different type, and vice versa.

In some embodiments, a UAV or other movable objects can be adapted tocarry one or more sensors. The one or more sensors may be configured tocollect relevant data, such as information relating to the UAV's state,the surrounding environment, or the objects and obstacles within theenvironment. The relevant data may be analyzed, processed, or be used infurther applications. For example, based on the relevant data that iscollected, it can be possible to generate control signals forcontrolling UAV navigation. Exemplary sensors suitable for use with theembodiments disclosed herein include location sensors (e.g., globalpositioning system (GPS) sensors, mobile device transmitters enablinglocation triangulation), vision sensors (e.g., imaging devices capableof detecting visible, infrared, or ultraviolet light, such as cameras),proximity or range sensors (e.g., ultrasonic sensors, lidar,time-of-flight or depth cameras), inertial sensors (e.g.,accelerometers, gyroscopes, inertial measurement units (IMUs)), altitudesensors, attitude sensors (e.g., compasses) pressure sensors (e.g.,barometers), audio sensors (e.g., microphones) or field sensors (e.g.,magnetometers, electromagnetic sensors).

Any suitable number and combination of sensors can be used, such as one,two, three, four, five, six, seven, eight, or more sensors. Optionally,the data can be received from sensors of different types (e.g., two,three, four, five, six, seven, eight, or more types). Sensors ofdifferent types may measure different types of signals or information(e.g., position, orientation, velocity, acceleration, proximity,pressure, etc.) and/or utilize different types of measurement techniquesto obtain data. For instance, the sensors may include any suitablecombination of active sensors (e.g., sensors that generate and measureenergy from their own energy source) and passive sensors (e.g., sensorsthat detect available energy). As another example, some sensors maygenerate absolute measurement data that is provided in terms of a globalcoordinate system (e.g., position data provided by a GPS sensor,attitude data provided by a compass or magnetometer), while othersensors may generate relative measurement data that is provided in termsof a local coordinate system (e.g., relative angular velocity providedby a gyroscope; relative translational acceleration provided by anaccelerometer; relative attitude information provided by a visionsensor; relative distance information provided by an ultrasonic sensor,lidar, or time-of-flight camera). In some instances, the localcoordinate system may be a body coordinate system that is definedrelative to the UAV.

The sensors can be configured to collect various types of data, such asdata relating to the UAV, the surrounding environment, or objects withinthe environment. For example, at least some of the sensors may beconfigured to provide data regarding a state of the UAV. The stateinformation provided by a sensor can include information regarding aspatial disposition of the UAV (e.g., location or position informationsuch as longitude, latitude, and/or altitude; orientation or attitudeinformation such as roll, pitch, and/or yaw). The state information canalso include information regarding motion of the UAV (e.g.,translational velocity, translational acceleration, angular velocity,angular acceleration, etc.). A sensor can be configured, for instance,to determine a spatial disposition and/or motion of the UAV with respectto up to six degrees of freedom (e.g., three degrees of freedom inposition and/or translation, three degrees of freedom in orientationand/or rotation). The state information may be provided relative to aglobal coordinate system or relative to a local coordinate system. Aglobal coordinate system may refer to a coordinate system independent toa location of the UAV or another entity. A local coordinate system mayrefer to a coordinate system relative to the UAV or another entity. Forexample, a sensor can be configured to determine the distance betweenthe UAV and the user controlling the UAV, or the distance between theUAV and the starting point of flight for the UAV. In some instances, asensor can be configured to determine the distance between the UAV andan object near the UAV.

The data obtained by the sensors may provide various types ofenvironmental information. For example, the sensor data may beindicative of an environment type, such as an indoor environment,outdoor environment, low altitude environment, or high altitudeenvironment. The sensor data may also provide information regardingcurrent environmental conditions, including weather (e.g., clear, rainy,snowing), visibility conditions, wind speed, time of day, and so on.Furthermore, the environmental information collected by the sensors mayinclude information regarding the objects in the environment, such asthe obstacles described herein or landmarks that are recognizable by aprocessor. Obstacle information may include information regarding thenumber, density, geometry, spatial disposition, movement, trajectory,and/or velocity of obstacles in the environment.

In some embodiments, the movable object can be configured to carry aload. The load can include one or more of passengers, cargo, equipment,instruments, and the like. The load can be provided within a housing.The housing may be separate from a housing of the movable object, or bepart of a housing for an movable object. Alternatively, the load can beprovided with a housing while the movable object does not have ahousing. Alternatively, portions of the load or the entire load can beprovided without a housing. The load can be rigidly fixed relative tothe movable object. Optionally, the load can be movable relative to themovable object (e.g., translatable or rotatable relative to the movableobject).

In some embodiments, the load includes a payload. The payload can beconfigured not to perform any operation or function. Alternatively, thepayload can be a payload configured to perform an operation or function,also known as a functional payload. For example, the payload can includeone or more sensors for surveying one or more targets. Any suitablesensor can be incorporated into the payload, such as an image capturedevice (e.g., a camera), an audio capture device (e.g., a parabolicmicrophone), an infrared imaging device, or an ultraviolet imagingdevice. The sensor can provide static sensing data (e.g., a photograph)or dynamic sensing data (e.g., a video). In some embodiments, the sensorprovides sensing data for the target of the payload. Alternatively or incombination, the payload can include one or more emitters for providingsignals to one or more targets. Any suitable emitter can be used, suchas an illumination source or a sound source. In some embodiments, thepayload includes one or more transceivers, such as for communicationwith a module remote from the movable object. Optionally, the payloadcan be configured to interact with the environment or a target. Forexample, the payload can include a tool, instrument, or mechanismcapable of manipulating objects, such as a robotic arm.

Optionally, the load may include a carrier. The carrier can be providedfor the payload and the payload can be coupled to the movable object viathe carrier, either directly (e.g., directly contacting the movableobject) or indirectly (e.g., not contacting the movable object).Conversely, the payload can be mounted on the movable object withoutrequiring a carrier. The payload can be integrally formed with thecarrier. Alternatively, the payload can be releasably coupled to thecarrier. In some embodiments, the payload can include one or morepayload elements, and one or more of the payload elements can be movablerelative to the movable object and/or the carrier, as described above.

The carrier can be integrally formed with the movable object.Alternatively, the carrier can be releasably coupled to the movableobject. The carrier can be coupled to the movable object directly orindirectly. The carrier can provide support to the payload (e.g., carryat least part of the weight of the payload). The carrier can include asuitable mounting structure (e.g., a gimbal platform) capable ofstabilizing and/or directing the movement of the payload. In someembodiments, the carrier can be adapted to control the state of thepayload (e.g., position and/or orientation) relative to the movableobject. For example, the carrier can be configured to move relative tothe movable object (e.g., with respect to one, two, or three degrees oftranslation and/or one, two, or three degrees of rotation) such that thepayload maintains its position and/or orientation relative to a suitablereference frame regardless of the movement of the movable object. Thereference frame can be a fixed reference frame (e.g., the surroundingenvironment). Alternatively, the reference frame can be a movingreference frame (e.g., the movable object, a payload target).

In some embodiments, the carrier can be configured to permit movement ofthe payload relative to the carrier and/or movable object. The movementcan be a translation with respect to up to three degrees of freedom(e.g., along one, two, or three axes) or a rotation with respect to upto three degrees of freedom (e.g., about one, two, or three axes), orany suitable combination thereof.

In some instances, the carrier can include a carrier frame assembly anda carrier actuation assembly. The carrier frame assembly can providestructural support to the payload. The carrier frame assembly caninclude individual carrier frame components, some of which can bemovable relative to one another. The carrier actuation assembly caninclude one or more actuators (e.g., motors) that actuate movement ofthe individual carrier frame components. The actuators can permit themovement of multiple carrier frame components simultaneously, or may beconfigured to permit the movement of a single carrier frame component ata time. The movement of the carrier frame components can produce acorresponding movement of the payload. For example, the carrieractuation assembly can actuate a rotation of one or more carrier framecomponents about one or more axes of rotation (e.g., roll axis, pitchaxis, or yaw axis). The rotation of the one or more carrier framecomponents can cause a payload to rotate about one or more axes ofrotation relative to the movable object. Alternatively or incombination, the carrier actuation assembly can actuate a translation ofone or more carrier frame components along one or more axes oftranslation, and thereby produce a translation of the payload along oneor more corresponding axes relative to the movable object.

In some embodiments, the movement of the movable object, carrier, andpayload relative to a fixed reference frame (e.g., the surroundingenvironment) and/or to each other, can be controlled by a terminal. Theterminal can be a remote control device at a location distant from themovable object, carrier, and/or payload. The terminal can be disposed onor affixed to a support platform. Alternatively, the terminal can be ahandheld or wearable device. For example, the terminal can include asmartphone, tablet, laptop, computer, glasses, gloves, helmet,microphone, or suitable combinations thereof. The terminal can include auser interface, such as a keyboard, mouse, joystick, touchscreen, ordisplay. Any suitable user input can be used to interact with theterminal, such as manually entered commands, voice control, gesturecontrol, or position control (e.g., via a movement, location or tilt ofthe terminal).

The terminal can be used to control any suitable state of the movableobject, carrier, and/or payload. For example, the terminal can be usedto control the position and/or orientation of the movable object,carrier, and/or payload relative to a fixed reference from and/or toeach other. In some embodiments, the terminal can be used to controlindividual elements of the movable object, carrier, and/or payload, suchas the actuation assembly of the carrier, a sensor of the payload, or anemitter of the payload. The terminal can include a wirelesscommunication device adapted to communicate with one or more of themovable object, carrier, or payload.

The terminal can include a suitable display unit for viewing informationof the movable object, carrier, and/or payload. For example, theterminal can be configured to display information of the movable object,carrier, and/or payload with respect to position, translationalvelocity, translational acceleration, orientation, angular velocity,angular acceleration, or any suitable combinations thereof. In someembodiments, the terminal can display information provided by thepayload, such as data provided by a functional payload (e.g., imagesrecorded by a camera or other image capturing device).

Optionally, the same terminal may both control the movable object,carrier, and/or payload, or a state of the movable object, carrierand/or payload, as well as receive and/or display information from themovable object, carrier and/or payload. For example, a terminal maycontrol the positioning of the payload relative to an environment, whiledisplaying image data captured by the payload, or information about theposition of the payload. Alternatively, different terminals may be usedfor different functions. For example, a first terminal may controlmovement or a state of the movable object, carrier, and/or payload whilea second terminal may receive and/or display information from themovable object, carrier, and/or payload. For example, a first terminalmay be used to control the positioning of the payload relative to anenvironment while a second terminal displays image data captured by thepayload. Various communication modes may be utilized between a movableobject and an integrated terminal that both controls the movable objectand receives data, or between the movable object and multiple terminalsthat both control the movable object and receives data. For example, atleast two different communication modes may be formed between themovable object and the terminal that both controls the movable objectand receives data from the movable object.

FIG. 10 illustrates a movable object 1000 including a carrier 1002 and apayload 1004, in accordance with embodiments. Although the movableobject 1000 is depicted as an aircraft, this depiction is not intendedto be limiting, and any suitable type of movable object can be used, aspreviously described herein. One of skill in the art would appreciatethat any of the embodiments described herein in the context of aircraftsystems can be applied to any suitable movable object (e.g., an UAV). Insome instances, the payload 1004 may be provided on the movable object1000 without requiring the carrier 1002. The movable object 1000 mayinclude propulsion mechanisms 1006, a sensing system 1008, and acommunication system 1010.

The propulsion mechanisms 1006 can include one or more of rotors,propellers, blades, engines, motors, wheels, axles, magnets, or nozzles,as previously described. For example, the propulsion mechanisms 1006 maybe rotor assemblies or other rotary propulsion units, as disclosedelsewhere herein. The movable object may have one or more, two or more,three or more, or four or more propulsion mechanisms. The propulsionmechanisms may all be of the same type. Alternatively, one or morepropulsion mechanisms can be different types of propulsion mechanisms.The propulsion mechanisms 1006 can be mounted on the movable object 1000using any suitable means, such as a support element (e.g., a driveshaft) as described elsewhere herein. The propulsion mechanisms 1006 canbe mounted on any suitable portion of the movable object 1000, such onthe top, bottom, front, back, sides, or suitable combinations thereof.

In some embodiments, the propulsion mechanisms 1006 can enable themovable object 1000 to take off vertically from a surface or landvertically on a surface without requiring any horizontal movement of themovable object 1000 (e.g., without traveling down a runway). Optionally,the propulsion mechanisms 1006 can be operable to permit the movableobject 1000 to hover in the air at a specified position and/ororientation. One or more of the propulsion mechanisms 1000 may becontrolled independently of the other propulsion mechanisms.Alternatively, the propulsion mechanisms 1000 can be configured to becontrolled simultaneously. For example, the movable object 1000 can havemultiple horizontally oriented rotors that can provide lift and/orthrust to the movable object. The multiple horizontally oriented rotorscan be actuated to provide vertical takeoff, vertical landing, andhovering capabilities to the movable object 1000. In some embodiments,one or more of the horizontally oriented rotors may spin in a clockwisedirection, while one or more of the horizontally rotors may spin in acounterclockwise direction. For example, the number of clockwise rotorsmay be equal to the number of counterclockwise rotors. The rotation rateof each of the horizontally oriented rotors can be varied independentlyin order to control the lift and/or thrust produced by each rotor, andthereby adjust the spatial disposition, velocity, and/or acceleration ofthe movable object 1000 (e.g., with respect to up to three degrees oftranslation and up to three degrees of rotation).

The sensing system 1008 can include one or more sensors that may sensethe spatial disposition, velocity, and/or acceleration of the movableobject 1000 (e.g., with respect to up to three degrees of translationand up to three degrees of rotation). The one or more sensors caninclude global positioning system (GPS) sensors, motion sensors,inertial sensors, proximity sensors, or image sensors. The sensing dataprovided by the sensing system 1008 can be used to control the spatialdisposition, velocity, and/or orientation of the movable object 1000(e.g., using a suitable processing unit and/or control module, asdescribed below). Alternatively, the sensing system 1008 can be used toprovide data regarding the environment surrounding the movable object,such as weather conditions, proximity to potential obstacles, locationof geographical features, location of manmade structures, and the like.

The communication system 1010 enables communication with terminal 1012having a communication system 1014 via wireless signals 1016. Thecommunication systems 1010, 1014 may include any number of transmitters,receivers, and/or transceivers suitable for wireless communication. Thecommunication may be one-way communication, such that data can betransmitted in only one direction. For example, one-way communicationmay involve only the movable object 1000 transmitting data to theterminal 1012, or vice-versa. The data may be transmitted from one ormore transmitters of the communication system 1010 to one or morereceivers of the communication system 1012, or vice-versa.Alternatively, the communication may be two-way communication, such thatdata can be transmitted in both directions between the movable object1000 and the terminal 1012. The two-way communication can involvetransmitting data from one or more transmitters of the communicationsystem 1010 to one or more receivers of the communication system 1014,and vice-versa.

In some embodiments, the terminal 1012 can provide control data to oneor more of the movable object 1000, carrier 1002, and payload 1004 andreceive information from one or more of the movable object 1000, carrier1002, and payload 1004 (e.g., position and/or motion information of themovable object, carrier or payload; data sensed by the payload such asimage data captured by a payload camera). In some instances, controldata from the terminal may include instructions for relative positions,movements, actuations, or controls of the movable object, carrier and/orpayload. For example, the control data may result in a modification ofthe location and/or orientation of the movable object (e.g., via controlof the propulsion mechanisms 1006), or a movement of the payload withrespect to the movable object (e.g., via control of the carrier 1002).The control data from the terminal may result in control of the payload,such as control of the operation of a camera or other image capturingdevice (e.g., taking still or moving pictures, zooming in or out,turning on or off, switching imaging modes, change image resolution,changing focus, changing depth of field, changing exposure time,changing viewing angle or field of view). In some instances, thecommunications from the movable object, carrier and/or payload mayinclude information from one or more sensors (e.g., of the sensingsystem 1008 or of the payload 1004). The communications may includesensed information from one or more different types of sensors (e.g.,GPS sensors, motion sensors, inertial sensor, proximity sensors, orimage sensors). Such information may pertain to the position (e.g.,location, orientation), movement, or acceleration of the movable object,carrier and/or payload. Such information from a payload may include datacaptured by the payload or a sensed state of the payload. The controldata provided transmitted by the terminal 1012 can be configured tocontrol a state of one or more of the movable object 1000, carrier 1002,or payload 1004. Alternatively or in combination, the carrier 1002 andpayload 1004 can also each include a communication module configured tocommunicate with terminal 1012, such that the terminal can communicatewith and control each of the movable object 1000, carrier 1002, andpayload 1004 independently.

In some embodiments, the movable object 1000 can be configured tocommunicate with another remote device in addition to the terminal 1012,or instead of the terminal 1012. The terminal 1012 may also beconfigured to communicate with another remote device as well as themovable object 1000. For example, the movable object 1000 and/orterminal 1012 may communicate with another movable object, or a carrieror payload of another movable object. When desired, the remote devicemay be a second terminal or other computing device (e.g., computer,laptop, tablet, smartphone, or other mobile device). The remote devicecan be configured to transmit data to the movable object 1000, receivedata from the movable object 1000, transmit data to the terminal 1012,and/or receive data from the terminal 1012. Optionally, the remotedevice can be connected to the Internet or other telecommunicationsnetwork, such that data received from the movable object 1000 and/orterminal 1012 can be uploaded to a website or server.

FIG. 11 is a schematic illustration by way of block diagram of a system1100 for controlling a movable object, in accordance with embodiments.The system 1100 can be used in combination with any suitable embodimentof the systems, devices, and methods disclosed herein. The system 1100can include a sensing module 1102, processing unit 1104, non-transitorycomputer readable medium 1106, control module 1108, and communicationmodule 1110.

The sensing module 1102 can utilize different types of sensors thatcollect information relating to the movable objects in different ways.Different types of sensors may sense different types of signals orsignals from different sources. For example, the sensors can includeinertial sensors, GPS sensors, proximity sensors (e.g., lidar), orvision/image sensors (e.g., a camera). The sensing module 1102 can beoperatively coupled to a processing unit 1104 having a plurality ofprocessors. In some embodiments, the sensing module can be operativelycoupled to a transmission module 1112 (e.g., a Wi-Fi image transmissionmodule) configured to directly transmit sensing data to a suitableexternal device or system. For example, the transmission module 1112 canbe used to transmit images captured by a camera of the sensing module1102 to a remote terminal.

The processing unit 1104 can have one or more processors, such as aprogrammable processor (e.g., a central processing unit (CPU)). Theprocessing unit 1104 can be operatively coupled to a non-transitorycomputer readable medium 1106. The non-transitory computer readablemedium 1106 can store logic, code, and/or program instructionsexecutable by the processing unit 1104 for performing one or more steps.The non-transitory computer readable medium can include one or morememory units (e.g., removable media or external storage such as an SDcard or random access memory (RAM)). In some embodiments, data from thesensing module 1102 can be directly conveyed to and stored within thememory units of the non-transitory computer readable medium 1106. Thememory units of the non-transitory computer readable medium 1106 canstore logic, code and/or program instructions executable by theprocessing unit 1104 to perform any suitable embodiment of the methodsdescribed herein. For example, the processing unit 1104 can beconfigured to execute instructions causing one or more processors of theprocessing unit 1104 to analyze sensing data produced by the sensingmodule. The memory units can store sensing data from the sensing moduleto be processed by the processing unit 1104. In some embodiments, thememory units of the non-transitory computer readable medium 1106 can beused to store the processing results produced by the processing unit1104.

In some embodiments, the processing unit 1104 can be operatively coupledto a control module 1108 configured to control a state of the movableobject. For example, the control module 1108 can be configured tocontrol the propulsion mechanisms of the movable object to adjust thespatial disposition, velocity, and/or acceleration of the movable objectwith respect to six degrees of freedom. Alternatively or in combination,the control module 1108 can control one or more of a state of a carrier,payload, or sensing module.

The processing unit 1104 can be operatively coupled to a communicationmodule 1110 configured to transmit and/or receive data from one or moreexternal devices (e.g., a terminal, display device, or other remotecontroller). Any suitable means of communication can be used, such aswired communication or wireless communication. For example, thecommunication module 1110 can utilize one or more of local area networks(LAN), wide area networks (WAN), infrared, radio, WiFi, point-to-point(P2P) networks, telecommunication networks, cloud communication, and thelike. Optionally, relay stations, such as towers, satellites, or mobilestations, can be used. Wireless communications can be proximitydependent or proximity independent. In some embodiments, line-of-sightmay or may not be required for communications. The communication module1110 can transmit and/or receive one or more of sensing data from thesensing module 1102, processing results produced by the processing unit1104, predetermined control data, user commands from a terminal orremote controller, and the like.

The components of the system 1100 can be arranged in any suitableconfiguration. For example, one or more of the components of the system1100 can be located on the movable object, carrier, payload, terminal,sensing system, or an additional external device in communication withone or more of the above. Additionally, although FIG. 11 depicts asingle processing unit 1104 and a single non-transitory computerreadable medium 1106, one of skill in the art would appreciate that thisis not intended to be limiting, and that the system 1100 can include aplurality of processing units and/or non-transitory computer readablemedia. In some embodiments, one or more of the plurality of processingunits and/or non-transitory computer readable media can be situated atdifferent locations, such as on the movable object, carrier, payload,terminal, sensing module, additional external device in communicationwith one or more of the above, or suitable combinations thereof, suchthat any suitable aspect of the processing and/or memory functionsperformed by the system 1100 can occur at one or more of theaforementioned locations.

As used herein A and/or B encompasses one or more of A or B, andcombinations thereof such as A and B.

While preferred embodiments of the present invention have been shown anddescribed herein, it will be obvious to those skilled in the art thatsuch embodiments are provided by way of example only. Numerousvariations, changes, and substitutions will now occur to those skilledin the art without departing from the invention. It should be understoodthat various alternatives to the embodiments of the invention describedherein may be employed in practicing the invention. It is intended thatthe following claims define the scope of the invention and that methodsand structures within the scope of these claims and their equivalents becovered thereby.

What is claimed is:
 1. An apparatus for detecting an obstacle, saidapparatus comprising: a plurality of passive infrared sensors on board amovable object and including: a first passive infrared sensor having afirst detection range and a first field of view; and two or more secondpassive infrared sensors each having a second detection range and asecond field of view, the second detection range being longer than thefirst detection range, and the second field of view being smaller thanthe first field of view; and one or more processors, collectively orindividually configured to: receive a signal from at least one passiveinfrared sensor of said plurality indicative of a heat signal from theobstacle; recognize the obstacle based on the heat signal; and determinewhether to effect a collision avoidance maneuver for the movable objectto avoid the obstacle.
 2. The apparatus of claim 1, wherein the one ormore processors are configured to determine whether to effect thecollision avoidance maneuver based on a speed and direction of themovable object and/or the obstacle.
 3. The apparatus of claim 1, whereinthe collision avoidance maneuver includes braking, altering a directionof a moving course of the movable object, stopping one or morepropulsion units of the movable object, deploying one or more airbagsand/or deploying one or more parachutes.
 4. The apparatus of claim 1,wherein a collective field of view of the plurality of infrared sensorscovers a 360 angle around the movable object.
 5. The apparatus of claim1, wherein the one or more processors are further configured to detect apeak value of an output signal of the passive infrared sensor andcalculate or determine a distance based on a calibration curve showing arelationship between distances and peak values.
 6. The apparatus ofclaim 5, wherein a distance from the movable object to the obstacle attime points subsequent to a first detection of the obstacle isdetermined.
 7. The apparatus of claim 1, wherein the first passiveinfrared sensor is located on an upper surface of the movable object. 8.The apparatus of claim 1, wherein each of the two or more second passiveinfrared sensors is located on one of lateral sides of the movableobject.
 9. The apparatus of claim 1, wherein: the first field of view ofthe first passive infrared sensor covers a 360° angle; and the secondfields of view of the two or more second passive infrared sensorscollectively cover a 360° angle.
 10. The apparatus of claim 1, wherein:the collision avoidance maneuver is a first collision avoidancemaneuver; and the one or more processors are further configured to:determine a distance to the obstacle based on the heat signal; determinewhether the distance is within a threshold distance; effect the firstcollision avoidance maneuver for the movable object to avoid theobstacle in response to determining that the distance is within thethreshold distance; and effect a second collision avoidance maneuver forthe movable object to avoid the obstacle in response to determining thatthe distance is outside the threshold distance, the second collisionavoidance maneuver being different from the first collision avoidancemaneuver.